DOCUMENT RESUME 

ED 377 478 CS 214 627 



AUTHOR 
TITLE 

REPORT NO 
PUB DATE 
NOTE 

PUB TYPE 
JOURNAL CTT 



EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Harnack, Andrew; And Others 

The Impact of Kentucky's Educational Reform Act on 
Writing throughout the Commonwealth. 
ISSN-0897-263X 
Dec 94 
6p . 

Reports - Descriptive (141) — Journal Articles (080) 
Composition Chronicle: Newsletter for Writing 
Teachers; v7 n8 p4-7 Dec 1994 

MF01/PC01 Plus Postage, 

Educational Change; Higher Education; "Portfolio 
Assessment; Portfolios (Background Materials); 
Secondary Education; Student Evaluation; Surveys; 
^Writing Achievement; Writing Across the Curriculum; 
^Writing Evaluation; ''Writing Instruction; Writing 
Research 

-Kentucky Educaticn Reform Act 1990; Writing 
Contexts 



ABSTRACT 

The central role of writing in Kentucky's Education 
Reform Act is most evident in Kentucky's new assessment system, which 
employs writing on all levels. Even tests that have repently included 
multiple-choice items may be replaced by response items that require 
students to apply knowledge, concepts, and skills in a writing 
format. Writing itself is being assessed through a portfolio system, 
whereby students are required to prepare and revise a collection of 
their best work to be presented for evaluation by a team of scorers. 
Portfolios must include a table of contents, a letter to the 
reviewer, a personal essay of some kind, and three pieces of an 
analytical nature. To measure the consequences of this new emphasis 
in writing across the curriculum for higher education, especially 
col 1 ege- level writing, a survey was conducted. Questionnaires were 
sent to 41 directors of composition and department chairs of 
Kentucky's post-secondary public and private institutions; 13 were 
returned. The survey focused on two areas relating to the reform act: 
(1) the use of portfolios for placement, instruction and assessment; 
and (2) changes in the preparation of freshman composition students. 
The results showed reason for caution. A small majority attributed 
positive changes to the reform act; others were not willing to note 
such changes because of a lack of evidence. They were somewhat 
skeptical because of the reform act's utilitarian agenda, its 
implicit demand that teachers conform to the needs of the corporate- 
culture, and the difficulty of scoring a large number of portfolios 
in a limited amount of time. <TB) 
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The Impact of Kentucky's Educational Reform Act 
on Writing throughout the Commonwealth 

Andrew Harnack, David Elias, Charles Whitakcr 
Eastern Kentucky University 



In 1990. by requirement of its highest court, Kentucky began 
a comprehensive transformation of its educational system. 
Described by PBS in America's Educational Revolution: A 
Report from the Frontline (aired April 23, 1993) as "the 
largest and most radical school reform" effort in the nauon to date, 
Kentucky's Educational Reform Act (KERA) has dramatically 
affcctedcurricuJum, assessment, teaching practices, and even school 
structure and scheduling. In this transformation, writing has played 
a central role — both in curriculum and assessment. All forms of 
Kentucky's new assessment system, which documents student and 
school progress, require students to write — and to write well. 
Indeed, an estimated eighty percent of the assessment overall is 
based upon students* writing. 

If. as U.S. Education Secretary Richard Riley has said, 
Kentucky's reform is "a lighthouse for the rest of the nation* 1 (qtd. 
Kentucky Teacher, February 1994), then information about ihe 
reform and the role of writing in this reform deserves attention. 
Furthermore, though KERA applies to the K - 12 system, what 
happens in that system clearly will influence higher education. 
Though recognizing that the reform is still in progress, we offer 
information about writing in Kentucky's educational reform at 
present and discuss some of the impact on writing throughout the 
commonwealth. 

Writing and Assessment 

The central role of writing in KERA is most evident in 
Kentucky's new assessment system. The assessment, designed in 
Kentucky with strong influence from teachers, follows Kentucky's 
outcome-based educational philosophy and emphasis on students' 
ability to demonstrate in alternate forms of testing that they know 
and can do what Kentucky's new Learning Goals and Academic 
Expectations require. Three forms of testing now arc in place: 
Transitional Tests (Open Response), grades 4, 8. and 11; Perfor- 
mance Events, grades 4. 8, and 11: and Portfolios, grades 4, 5 
^ (mam). 8, and 12. In each form of testing, students must write, and 
^ an interdisciplinary writing portfolio focuses specifically upon 
^ writing. 

^ Although pastTransitional Tests have included multiple-choice 
— items, plans are to phase out these items and rely upon open- 
rfi response items that require students to apply knowledge, concepts. 
^ and skills. For example, one released open-response item presents 
a pic graph that indicates different percentages of income for a 
budget. Students arc to analyze the budget and write to explain how 
they would modify it in order to purchase a car, For open-response 
tests in all study areas, students mast do far more than choose a best 
answer or complete one-sentence answers. They must write. As a 



result, many teachers now require more writing, including such 
practices as learning logs and response journals, to help students 
learn and prepare for open -response tests. 

Performance events present students with some task thai 
requires them to demonstrate that they can apply information, 
skills, and core concepts. Students might, for example, be given a 
task to analyze a liquid (materials and such are provided) and to 
write about their findings or about the processes they followed. 
Though some performance tasks require students to work 
collaboratively, individual performance also is tested. Test makers 
develop tasks to address the Learner Goals and Academic Expec- 
tations, and these tasks also address realistic or "authentic" situa- 
tions and problems students will face in the workplace and in their 
personal lives. For these assessment tasks in all study areas, writing 
often is the means by which students demonstrate what they know 
and can do. 

Portfolios 

Kentucky has made a major commitment id portfolio assess- 
ment, including a writing portfolio. Working with a consul ting firm. 
Advanced Systems, located in New Hampshire, the Kentucky 
Department of Education and teachers across the commonwealth 
designed the portfolio system that requires students to collect a 
variety of samples of their writing, with representation from differ- 
ent study areas. 

Six samples arc required from cac* student. The overall 
expectation for all Kentucky schools is that students will write with 
a variety of purposes in a variety of forms for a variety of audiences. 
Though assessment portfolios are collected at targeted grades, 
working portfolios arc maintained at other grades as well and in 
some districts arc passed from grade to grade. Here, for example, 
arc the requirements for a portfolio submitted by a grade-twelve 
student 

• Tabic of Contents including description of each entry, the 
study area for which it is written, and the page number in the 
portfolio. 

• Letter to Reviewer in which the student analyzes himself/ 
herself as a writer and reflects on the pieces in the portfolio. 

• One of the following: 

a. Personal Narrative 

b. Memoir 

c. Personal Essay 

• or • 

d. Personal Vignette 

• One Short Story, Poem, or Play/Script 
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• Three pieces, each of which will achicvconcor moreof these 
purposes: 

a. Predict an Outcome 

b. Defend a Position 

c. AnaJyze or Evaluate a Situation, Person, Place, or Thing 

d. Solve a Problem 

e. Explain a Process or Concept 

f. Draw a Conclusion 

g. Create a Model 

Designing and implementing die oring process for writing 
portfolios have been major endeavors. Last year, 137,561 writing 
portfolios were scored, involving teachers throughout the stale. The 
assessment includes the following; 

• collection of samples at targeted grade levels 

• a Scoring Guide that indicates criteria and performance 
levels 

• sets of Benchmarks for the four performance levels 

• a holistic scoring process using trained scorers 

• a series of consistency checks by Kentucky teachers and 
outside scorers 

• reports to improve instruction and performance 

Overall reliability of the assessment system, of course, is a 
concern. A perfect reliability index is 1 . Kentucky's goal is .90 for 
school-level data. For 199 1 - 92, reliability measures were between 
.85 and .90. For 1992- 93. at all grade levels, results exceeded .90 
(Kentucky Department of Education, Kentucky Teacher, August 
1994). Scores of assessment portfolios arc reported to schools to 
improve instruction, student performance, and school accountabil- 
ity. Indeed, "sanctions and rewards/* including money for schools, 
arc determined in pan by these scores. The role of writing in 
Kentucky's schools is an important one for all teachers and adm in- 



istrators. 

In addition to providing scoring criteria, instructional analysis 
and other information, the Holistic Scoring Guide describes four 
levels of performance; novice, apprentice, proficient, and distin- 
guished. The most recent results of scoring (Figure 1 ) indicate that 
standards are high, and though educators would be delighted if 
many students demonstrated a "distinguished" level of perfor- 
mance, few students reach this level. 

Although many students still perform at or below the Appren- 
tice level in writing, increases at the Proficient level for the writing 
portfolio areencouraging. Since writing plays a key role in all forms 
of assessment, overall trends deserve attention. Thomas Boysen, 
Commissioner of Education, reports that 'The trend data from 1 992 
to 1993 showed an 8 percent increase in student achievement in 
reading, writing, mathematics, science and social studies" (Ken- 
tucky Department of Education, "Measuring Up" 1994). 

The impact of the role of writing in the new assessment in 
Kentucky has been strong, including writing across the curriculum; 
portfolios at elementary, middle, and secondary grades; increased 
staff development for teachers to help them use and score writing; 
integration of writing and learning; and gradual improvement in 
writing and in learning. If these trends continue, what ?re the 
consequences for higher education, especially college-level writ- 
ing programs? 

The Results of a Survey- 
To help answer this question wc majlcd in die Fall of 1993 a 
survey to all composiuon directors and department chairs at 41 of 
Kentucky's post-secondary public and private institutions; 1 3 were 
returned. The survey focused on two area* relating to KERA: (A) 
the use of portfolios for placement, instruction and assessment, and 
(B) changes in the preparation of freshman composiuon students. 



Figure 1. 

Assessment 
of Portfolios, 
1991-92 
and 1992-93 



(Prc-audit averages of 
grades 4, 8. 12. 

Source; Kentucky Department 
of Education, Division of 
Assessment Implementation, 
10-28-94.) 
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The results provided an opportunity to reflect on school reform in 
general and on the impact of KERA's reforms for writing instruc- 
non m particular. 

Reasons for caution 

The respondents were cautious in estimating the effects of 
KERA on their incoming freshmen. A small majority attributed 
positive changes to KERA — more experience with writing, writing 
across the curriculum, and group work. Others did not note such 
changes or were not willing to attribute them to KERA because of 
lack of evidence. Nor were any changes in ihcir departments 
attributable to KERA; many respondents said what they already do 
will work on incoming students. The survey revealed that less than 
twenty- five percent of respondents use portfolios, only for instruc- 
tion; no respondent indicated departmental use of portfolios for 
assessment The generally positive view of KERA was tempered; 
in several cases, respondents perceived the following as problems: 

• large classes 

• the high cost for training and implementing programs 

• confusion about standards 

• heavy teaching loads 

• lack of training 

• lack of time to practice before the big pay-off assessments 
In addition, since KERA's portfolio assessment of graduating 

seniors is not available for admission purposes, universities have 
not been able to use these scores for placing students in appropriate 
writing courses. Considering such problems, academics have two 
good reasons to be cautious about KERA's impact, both of which 
came out in follow-up interviews with selected respondents. The 
first concerns the pragmatic difficulty of translating plans for 
reform in to teaching, learning and assessment practices. The 
second has to do with what one respondent called KERA's ^utilitar- 
ian agenda" and its attempts to bridge gaps between school and the 
workplace. 

As several people pointed out in their responses, KERA 
adulates many of the suggestions that composition teachers and 
researchers have been making in their journal articles, conference 
papers, and workshops for years: use more group work, practice 
writing more frequently and in more varied settings, give tasks that 
more closely resemble those of workers. By means of feedback 
from the K-12 system, academics now get to see how these 
suggestions work on a large scale in real schools with thousands of 
real students. That translation should be reason for concern in itself 
because no curriculum can be teacher-proof, that is, immune to 
alterations. When actual human beings try to put the profession's 
brilliant ideas on how to teach into practice, they interpret, reacting 
to local conditions and specific classroom dynamics, and. for belter 
or worse, change these ideas. Such distortion in implementing 
reform is inevitable. 

The second reason to be cautious, some say, is KERAs 
emphasis to better prepare students for the workplace. The benefits 
of Kentucky's reform are tempered, some academics contend, by 
the need to question KERA's apparent messages that students and 
teachers conform to the values and expectations of corporate 
cultures, thai is, to the tenets of corporate capitalism. 

The assumption that what works at one level of schooling will 
work at other levels also needs to be interrogated. It seems reason- 
able to assume, for example, that since college students collaborate 



successfully, so too, can primary and middle schoolers. What 
students collaborate on and how they collaborate may be different 
at different levels, however, dii Cerent enough to render compari- 
sons, and Translations, diff icult. Furthermore, what works for K- 3 2 
teachers may not work for postsecondary educators. For some 
respondents this assumption, mat schools, colleges, and the work- 
place should become one seamless system — decomextualtzed, 
ahistorical. univcrsalistic — presents the biggest obstacle to the 
successful impact of KERA in post- secondary education. It is not 
only impossible but questionable to try to create such a seamless 
system. 

Still, all the respondents found something to be happy with, and 
none predicted that students would be getting worse. Certainly, 
strengthening the connections among the public school sy stem, the 
higher education system, and the needs of employers has many 
benefits for students and employers. Under the terms of state 
budgeting for higher educauon. colleges and universities also 
benefit to the extent that they can turn out students that employers 
find satisfactory. This is a very attractive situation for many people, 
especially in light of budget shortages for the past se veral years and 
the foreseeable future. Whether this approach to coordinating 
schools and job markets will lead to better writing or simply to a 
ne* set of literate activities subordinated to corporate interests, or 
some odd mixture of both, remains to be seen. 

Portfolios at Eastern Kentucky University 

Even prior to KERA's initiatives, our faculty recognized a need 
to improve how students write and initiated some changes similar 
to those advocated by KERA. The need for such change is indicated 
inasmuch as one out of four students cannot satisfy a junior-level 
writing requirement For some this failure rate is a mark of rising 
standards at the university; for others it is a sign that the university 
is not doing a good job in teaching writing. However one interprets 
the results, it is still apparent that many students need to improve 
their writing ability. KERA's emphasis on the need for change to 
improve writing should be continued in post- secondary education. 

Although thankful for KERA's emphasis on writing, we have 
come to realize that it is difficult to respond with significant change 
at a regional university. Predictably, some faculty see any intrusion 
into their classrooms as an invasion of their academic freedom. 
Moreover, assessments (no matter what the instruments — portfo- 
lios or otherwise) can always be perceived as problematic. How- 
ever, our university has been able to establish new directions, some 
reform, better pedagogies, and the possibility of more reliable 
assessments. 

Three years ago. we began our Pilot Portfolio Project which 
has led to a revision of syllabi and related classroom instruction. 
Using KERA's 1 2lh grade portfolio as a base, we have constructed 
our freshman writing program so that it takes advantage of the 
benefits of portfolio- based instruction. Our students now write for 
purposes similar to those required by KERA. Students present three 
finished pieces of their best work; they prepare die presentation of 
thcsepicccs with a letter of reflection. Thechange for students is not 
as great as that experienced by our instructors, most of whom are 
new to portfolio-driven instruction. Here, for example, are com- 
ments from one instructor describing the changes that have taken 
place within his classroom: 

My perception is dial the KERA-inspircd portfolio method 
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of teaching composition has transformed traditional pedagogy in 
a fundamental way. Before portfoIio[s] I was, I believe, a by-the- 
book son of teacher who allowed the textbook arrangement of 
subject matter to lead me in the sequence of assignments and 
topics discussed in class. Admittedly this procedure was not the 
most imaginative and it often resulted in my hiding behind the 
textbook arrangement and explanation of the subject matter, 
depending upon it for my authority (or part of it), and being 
hlmdcd 10 other avenues of approaching the topic at hand. In 
contrast, the portfolio approach docs not permit me the luxury of 
such defenses, dependencies, or bl indncss because, for better or 
worse, I have to come to grips with the elements of the lesson the 
students don't grasp immediately. 

Formerly ii was relatively easy for students themselves to 
use [this way of teaching] as an insuperable formality and ashield 
between the teacher and themselves. Now we all work more as 
equals at a process, at mastering a skill than we did before. I am 
not saddled with the burden of infallibility and am free to admit 
that I am still learning to write, too — as I assume we all are. 

The portfolio method allows the teacher to respond to 
individual needs as well as to the class as a whole and to keep a 
better balance in so doing than the prior traditional approach 
afforded. 

This response is typical of those made by our writing instruc- 
tors, and we have had some success in turning our program toward 
a wide-spread use of portfolios. But again, what's important is that 
this movement toward more use of portfolios is both in response to 
KERA and to recommendations of the profession nationally. 

KERA asks that we rethink large-scale assessments of student 
writing. To this end we also are seeking ways to develop a local, 
valid, and reliable instrument for students, instructors, and depart- 
ment evaluations. Yet we have a problem. We realize that our 
current end-of-the- semester proficiency essay examination, even 
v>hen graded holistically, carries all of the liabilities of a single 
instrument barrier/gateway examination. Regardless of our sympa- 
thy for portfolios, we have not figured out how to assess 2000 
portfolios at the end of a semester. Unlike our colleagues in 
Kentucky's high schools, we do not have the benefit of collecting 
portfolios in February and scoring them during the months that 
follow. Our registrar requires mat we have all grades in before the 
semester ends. Is it possible to score 2000 portfolios within a very 
short period of time? We hope so. If. however, some reasonably 
quick method of assessing portfolios is not possible, the use of 
pL \)lios may be difficult to sustain. Nevertheless, following 
KERA's lead, we are convinced that we want to encourage the use 
of portfolios. 

KERA has indeed been an agent of transformation, leading 
students lo write more effectively. Kentucky's reform demands that 
its educators and students commi I Ihcm sel ves to higher, measurable 
outcomes, showing through performance thai students arc in fact 
prepared for personal and vocational success. To what degree such 
reforms should and can influence Kentucky's college and univer- 
sity instruction is a question academics continue to raise even as 
they take heart in what has already been accomplished. 



David Elias coordinates the developmental English programs 
at EKU. He has taught writing to lawyers, engineers, computer 
scientists, medical researchers, and other working people. 

Charles Whitaker is Director of the Eastern Kentucky Writing 
Project He has conducted over a hundred KERA workshops for 
teachers to help them write and teach writing. £» 



Andrew Harnack (caches composition courses at all levels as 
well as courses in rhetorical theory. He is currently the Writing 
Program Administrator at EKU. 
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